feat(unofficial): mock 950dt + huawei-mindie for unofficial runs#257
Open
Oseltamivir wants to merge 12 commits into
Open
feat(unofficial): mock 950dt + huawei-mindie for unofficial runs#257Oseltamivir wants to merge 12 commits into
Oseltamivir wants to merge 12 commits into
Conversation
Adds Huawei Ascend 950DT GPU and Huawei MindIE framework so the Mock-ascend GitHub Actions run (25014782858) can be loaded via the unofficial-run viewer. TDP/power/cost are placeholder 9.99 — Huawei specs are not yet finalized. - HW_REGISTRY: new 950dt entry, vendor=Huawei, arch=Ascend, sort=9 - VENDOR_OKLCH_ZONES / VENDOR_HSL_ZONES: amber/yellow zone for Huawei (steals 30-60° HSL from AMD's lower band; AMD keeps 300-360 + 0-30) - FW_REGISTRY: huawei-mindie → "Huawei MindIE" - dynamic-colors Vendor type extended with 'huawei' - chart-utils BANNED_HUE_TEST/PREFERRED_ZONE entries for huawei (avoid AMD-red and NVIDIA-green clashes; preferred amber/yellow zone) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The Mock-ascend workflow uploads one bmk_<config>_conc<N>_<gpu>_<idx> artifact per concurrency instead of a single aggregated results_bmk. The unofficial-run API was filtering strictly on name === 'results_bmk', returning 404 even though the per-config artifacts contained valid rows. When results_bmk is absent, gather every artifact whose name starts with 'bmk_', download each, and concat the rows before normalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Make it able to render 8192 input tokens, 256 output tokens, like from runs/25457884421 |
Contributor
|
Claude finished @Oseltamivir's task in 7m 56s —— View job • Done — registered the 8K/256 sequence (8192/256)Run 25457884421 is the InfiniteBench DSv4 sweep that emits What changed
The flow that now works for
Checks
Not done
Todos
· branch: |
Contributor
Author
|
Make it able to render 8192 input tokens, 256 output tokens, like from runs/25457884421 Use this PR/branch, don't create a new branch |
Contributor
|
Claude finished @Oseltamivir's task in 4m 23s —— View job Done — registered 8K/256 sequence (8192/256) on this PR branchPushed commit Changes
The flow that now works for
Checks
Not done — needs your verification
|
Registers '8k/256' (isl=8192, osl=256) so unofficial-run overlays from sweeps like InferenceX run 25457884421 render instead of being silently dropped by islOslToSequence returning null. - packages/constants/src/models.ts: add 8k/256 to both conversion maps. - packages/app/src/lib/data-mappings.ts: add Sequence.EightK_256 with SEQUENCE_CONFIG entry (label "8K / 256"). Replace the binary 1k-vs-8k ternary in getModelAndSequenceFromArtifact with islOslToSequence so future ISL/OSL pairs registered in constants are picked up automatically rather than mis-mapped. - Tests added for round-trip + label coverage in models.test.ts / models-mapping.test.ts and a dsv4 + 8192/256 case in data-mappings.test.ts. Co-authored-by: Bryan Shan <Oseltamivir@users.noreply.github.com>
Contributor
Author
|
For unofficial runs, add a toggle where the dashboard treats them as ingested runs.
|
Contributor
|
Claude finished @Oseltamivir's task in 24m 55s —— View job Done — "Show as ingested" toggle for unofficial runsPushed How it works
What changed
Analytics: Browser verification (Playwright)Walked through the user-facing flow against
Checks
|
…ries
Adds a "Show as ingested" switch in the inference scatter legend (visible
when an unofficial run is loaded) that promotes overlay rows to first-class
points so they participate in the same filter pipeline as DB-ingested data:
hardware toggles, Optimal-only, precision filtering, and per-(hwKey, run)
roofline grouping.
Each (run, original hwKey) pair is rewritten to a synthesized hwKey of the
form `${origHwKey}__uorun${runId}` — preserving the GPU base prefix so
`getModelSortIndex` and `isKnownGpu` keep working — with a synthesized
HardwareEntry whose label embeds the run branch and a per-run color override
drawn from the existing overlay palette. Multiple runs and per-run multi-GPU
configs each surface as their own legend entry.
URL-synced via `i_uoff_ingested=1` so the state survives reloads and shares.
The X-shape overlay layer is suppressed and stale DOM is cleaned up when the
toggle flips on (the layer system has no built-in teardown for layers that
drop out of the array). Unit tests cover synth-key encoding, no-op behavior
when no overlay matches, and multi-run separation.
Co-authored-by: Bryan Shan <Oseltamivir@users.noreply.github.com>
Contributor
Author
|
for the previous edit, the results look good, but continue to build on it:
|
Contributor
|
Claude finished @Oseltamivir's task in 12m 4s —— View job Done — bare GPU labels + vendor-zone colors for ingested unofficial rowsPushed What changed
The wiring ( Browser verification (Playwright)Loaded
Each NVIDIA GPU sits in the 121.7–168.3 hue band — a different shade per GPU, exactly the spec. No legend label contains "infinitebench-..." or "mock-ascend-..." anywhere. Switched to 8K / 256 to confirm the InfiniteBench overlay ( TestsAdded 3 integration cases to
Updated the existing label/color assertions to expect bare GPU labels and an empty Checks
|
Drops the branch suffix from synth-key labels and the per-run color override so unofficial rows promoted via "Show as ingested" share the same vendor-aware palette as official rows. Two NVIDIA GPUs from one unofficial run now get distinct shades of green instead of one shared overlay-palette red. The branch + run URL still surface in `gpu` so the row tooltip retains provenance. Co-authored-by: Bryan Shan <Oseltamivir@users.noreply.github.com>
Contributor
Author
Contributor
|
Claude finished @Oseltamivir's task in 8m 40s —— View job Done — temp default to
|
… on unofficial-run load Temporary branch-only override: when a `?unofficialrun=` URL loads and the user did not pin `i_seq` / `i_metric` themselves, default the sequence to `8K / 256` and the y-axis metric to `Output Token Throughput per GPU` so InfiniteBench-style sweeps land on a useful default view alongside the model auto-switch from #243. Mirrors the dedupe shape of `computeAutoSwitchDecision` via a new pure helper `computeUnofficialOverrideDecision` so manual user picks stick once URL-synced and a fresh run-set transition can re-arm the override. Co-authored-by: Bryan Shan <Oseltamivir@users.noreply.github.com>
Oseltamivir
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
Author
|
Validated the offline B300 TRT path end to end.
Current preview: https://inferencemax-app-git-claude-huawei-950dt-0f2683-semianalysisai.vercel.app/inference?unofficialrun=27482213487 Validation: full unit suite 2,016 tests passed; typecheck, lint, and format checks passed. |
Contributor
Author
|
Allow for selection of multiple sequence length options on one graph. (e.g. 1k1k with 8k1k) |
Contributor
|
Claude finished @Oseltamivir's task in 25m 15s —— View job Done — multi-select ISL/OSL on the inference chartPushed How it works
Tests
Browser verification (Playwright on
|
… one chart The sequence picker becomes a multi-select. Picking >1 (e.g. 1K/1K + 8K/1K) overlays both shapes on the same axes — each (GPU, sequence) becomes its own legend line via a `__seq<compact>` hwKey suffix, so vendor-zone colors still distinguish hardware while the legend caption reads e.g. "B200 — 1K / 1K" vs "B200 — 8K / 1K". State: new `extraSequences: Sequence[]` lives in InferenceContext (URL param `i_seq_extra`, empty default). Primary `selectedSequence` semantics are unchanged for the calculator / evaluation / trends tabs. Pipeline: `useChartData` filters rows by `IN [primary, ...extras]`, groups by sequence, runs `transformBenchmarkRows` per sequence, and rewrites each output point's hwKey with `makeSeqSynthKey`. Synth hardware entries get the sequence appended to their label. `filterByGPU` is widened to strip the `__seq` suffix so the GPU selector still matches across sequences. Unofficial-run merge: when `mergeAsIngested` is on with multiple sequences, the merger iterates over (primary + extras), composing `base__seq<compact>__uorun<id>` synth keys so a (run, GPU, sequence) triple gets its own legend line. `parseSynthHwKey` keeps working because the seq delimiter precedes the run delimiter. Legend reset effect now also keys off `extraSequences` and `mergeAsIngested` so newly-introduced synth hwKeys become active when the user flips either toggle — fixes the Cursor "Merged series stay legend-inactive" finding on the prior commit where `reconcileActiveSet` only removed stale keys. Tests: - `sequence-synth-key.test.ts` (new) — round-trip, base prefix preservation, label tagging, compose with `__uorun`. - `unofficial-merge.test.ts` — multi-sequence merge produces `__seq__uorun` shape; primary-only with duplicate extra dedup is a no-op. - `useChartData.test.ts` — `filterByGPU` strips `__seq` suffix. Verified in Playwright at localhost:3000/inference: legend renders e.g. "GB300 NVL72 — 1K / 1K (Dynamo SGLang)" alongside the 8K / 1K variant, URL state restores from `?i_seq_extra=`, and overlay composition works with `?unofficialrun=27482213487&i_seq_extra=8k%2F1k&i_uoff_ingested=1` (B300 — 8K / 625 from the run + B300 — 8K / 1K official both render). Co-authored-by: Bryan Shan <Oseltamivir@users.noreply.github.com>
Contributor
Author
|
@claude merge main in, and resolve or address the PR reviews |
Contributor
|
Claude finished @Oseltamivir's task in 15m 41s —— View job Done — merged
|
| # | Finding | Severity | Status |
|---|---|---|---|
| 1 | Blind 8k/256 hides unofficial rows (GlobalFilterContext.tsx) |
Medium | ✅ Already addressed in 7b4dc87 — selectUnofficialDefaultSequence(unofficialAvailable, selectedModel, urlModel) only returns EightK_256 when it's in the run's available sequences for the target model, otherwise falls back to a sorted available sequence. Verified the merged code still has the same logic. |
| 2 | Merged series stay legend-inactive (InferenceContext.tsx:316) |
High | ✅ Already addressed in 747f1ad — the hw-reset key now includes `${extraSequencesKey} |
| 3 | Popstate skips ingested toggle sync (unofficial-run-provider.tsx) |
Low | ✅ Fixed in this turn — load now takes an isPopstate flag and re-reads params.get('i_uoff_ingested') on browser back/forward. |
| 4 | Merge toggle cleared after URL hydration (unofficial-run-provider.tsx) |
Medium | ✅ Fixed in this turn — both the useState initializer and the hydration useEffect now use readUrlParams() (the in-memory snapshot captured at module load, before url-state.ts's deferred cleanup), so share links with ?i_uoff_ingested=1 no longer race against URL cleanup. |
Why a single fix can't cover both popstate (#3) and hydration (#4)
- On initial mount, the URL may have already been stripped by
url-state.ts'ssetTimeout(0)cleanup, so the live URL is unreliable — we must read the in-memory snapshot. - On popstate, the browser has restored the previous history entry, so the live URL is the source of truth — and the snapshot is stale.
- The
load(isPopstate)flag picks the right source per call.
Why I didn't add a unit test for the popstate path
- The existing
unofficial-run-provider.test.tsonly exercises the pure helpers (buildChartData,parseAvailableModelsAndSequences). The provider itself isn't rendered there, and adding a render harness for one regression case felt out of scope. The behavior is exercised by browser verification + the logic is small.
Checks
-
pnpm typecheck— clean -
pnpm lint— 0 errors (also cleaned up 2 pre-existing nits inunofficial-merge.test.tsthat I touched) -
pnpm --filter @semianalysisai/inferencex-app exec vitest run— 2090/2090 passed -
pnpm --filter @semianalysisai/inferencex-constants exec vitest run— 30/30 passed - Browser-verified with Playwright that
?unofficialrun=…&i_uoff_ingested=1hydrates witharia-checked="true"on thescatter-uoff-as-ingestedswitch, even afterurl-state.tsruns its deferred URL cleanup. (Did not browser-test popstate end-to-end — sandboxed Playwright + dev mode makes history mutation testing brittle; the popstate code path is small and the logic is straightforward.)
Merge-conflict resolutions (commit 703c6b9)
9 files had textual conflicts — all were "both-sides-added" merges, no semantics had to be reconciled:
packages/constants/src/framework-aliases.ts—huawei-mindie+mooncake-atomboth registeredpackages/app/cypress/support/mock-data.ts—hwColorOverrides/extraSequences+compareGpuPairboth in mock contextpackages/app/src/app/api/unofficial-run/route.test.ts— both new test cases preservedpackages/app/src/lib/data-mappings.ts— both new imports preservedpackages/app/src/components/inference/types.ts— both new context fields preservedpackages/app/src/components/inference/hooks/useChartData.ts—getHardwareConfig+hardwareKeyMatchesAnyBaseboth imported;extraSequences+compareGpuPairboth parameterspackages/app/src/components/inference/InferenceContext.tsx— switchedhasMtpEngineExclusion→getModelExclusion(master refactored exclusion API); keptSequenceimport forextraSequences; threaded botheffectiveExtraSequencesandcompareGpuPairintouseChartDataand contextpackages/app/src/components/inference/ui/ChartDisplay.tsx— bothmergeAsIngestedandactiveOverlayHwTypesdestructuredpackages/app/src/components/inference/ui/ScatterGraph.tsx— both the "Show as ingested" toggle and master's reshaped log-scale toggle preserved
· branch:claude/huawei-950dt-mindie-unofficial
…-mindie-unofficial # Conflicts: # packages/app/cypress/support/mock-data.ts # packages/app/src/app/api/unofficial-run/route.test.ts # packages/app/src/components/inference/InferenceContext.tsx # packages/app/src/components/inference/hooks/useChartData.ts # packages/app/src/components/inference/types.ts # packages/app/src/components/inference/ui/ChartDisplay.tsx # packages/app/src/components/inference/ui/ScatterGraph.tsx # packages/app/src/lib/data-mappings.ts # packages/constants/src/framework-aliases.ts
Two related Cursor findings on the `i_uoff_ingested` URL param: 1. **Merge toggle cleared after URL hydration** — the post-mount sync was reading `window.location.search`, but `url-state.ts` strips share-link params on a deferred `setTimeout(0)`. If that cleanup ran before our effect, a shared link with `i_uoff_ingested=1` would silently turn the toggle off. Both the `useState` initializer and the hydration effect now read `readUrlParams()` (in-memory snapshot captured before cleanup). 2. **Popstate skips ingested toggle sync** — the `popstate` listener re-loaded run data but never re-read `i_uoff_ingested`. Browser back/forward could leave the toggle out of sync with the URL. The load function now takes an `isPopstate` flag and re-syncs the toggle from `window.location.search` only on popstate (not on the initial call, which would race with the cleanup above). Also: drop a stale lint nit in `unofficial-merge.test.ts` by switching the hue-extraction regex to a named-capture-group + unicode flag. Co-authored-by: Bryan Shan <Oseltamivir@users.noreply.github.com>
|
Found 8 test failures on Blacksmith runners: Failures
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c9c31a7. Configure here.
| // official series via InferenceContext, so suppress the X-shape overlay layer. | ||
| const overlayDataByChartType = useMemo(() => { | ||
| if (!unofficialRunInfo || !getOverlayData) { | ||
| if (mergeAsIngested || !unofficialRunInfo || !getOverlayData) { |
There was a problem hiding this comment.
Overlay ignores extra sequences
Medium Severity
With unofficial overlay mode (not “Show as ingested”), overlay points are loaded only for the primary selectedSequence. Extra ISL/OSL shapes from extraSequences appear in official useChartData series but never in the X-shape overlay, so unofficial points for those sequences are missing on the same chart.
Reviewed by Cursor Bugbot for commit c9c31a7. Configure here.
| const url = new URL(window.location.href); | ||
| if (v) url.searchParams.set('i_uoff_ingested', '1'); | ||
| else url.searchParams.delete('i_uoff_ingested'); | ||
| window.history.replaceState({}, '', url); |
There was a problem hiding this comment.
Share URL drops ingested toggle
Medium Severity
Toggling “Show as ingested” updates the address bar via history.replaceState but does not call writeUrlParams, so i_uoff_ingested never enters currentState. buildShareUrl only serializes currentState (plus unofficial run ids from the live URL), so generated share links omit the merge-as-ingested setting even when the toggle is on.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit c9c31a7. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unblocks loading the Mock-ascend run 25014782858 via the unofficial-run viewer (
?unofficialrun=25014782858).The mock artifacts use:
hw: 950dt— new GPU, Huawei Ascend 950DT (new vendor)framework: huawei-mindie— new frameworkmodel: deepseek-ai/DeepSeek-V4-Pro— already supported (dsv4)precision: fp4, sequences8K/1K,1K/1K— already supportedWithout these mappings,
hwToGpuKey('950dt')returnsnulland rows are silently skipped by the ETL → the unofficial-run viewer shows nothing.Changes
packages/constants/src/gpu-keys.tsHW_REGISTRY['950dt']— vendorHuawei, archAscend, labelAscend 950DT, sort9. TDP/power/cost set to placeholder9.99— Huawei figures are not yet finalized; bump to real values when available.huaweicolor zone inVENDOR_OKLCH_ZONES(amber 60–90°) andVENDOR_HSL_ZONES(amber 30–60°). To make room in the HSL wheel, AMD's lower band shrinks from 0–60 to 0–30 and NVIDIA's band shrinks from 60–195 to 90–195. AMD/NVIDIA still get >90° each.packages/constants/src/framework-aliases.tsFW_REGISTRY['huawei-mindie']→ labelHuawei MindIE.packages/app/src/lib/dynamic-colors.tsVendortype now includes'huawei';getVendor()recognizes theHuaweivendor string.packages/app/src/lib/chart-utils.tsBANNED_HUE_TEST/PREFERRED_ZONEhuawei entries — avoids AMD-red (320–40°) and NVIDIA-green (120–195°) in CIELab; prefers amber/yellow (50–95°).Notes for review
tput_per_gpu: 510.2041,median_ttft: 0.0, etc. — fake values.9.99and tune the chart sort order.Test plan
pnpm typecheckpnpm lintpnpm test:unit(1896 tests pass across all packages)?unofficialrun=25014782858— confirm 950dt overlay points render under DeepSeek-V4-Pro at 1K/1K, 8K/1K, etc., with conc 16/64/128.🤖 Generated with Claude Code
Note
Medium Risk
Touches chart data keys, legend reset, URL state, and unofficial-run loading across several providers; behavior is well covered by unit tests but preview validation for the target run is still important.
Overview
Adds Huawei Ascend 950DT (
950dt) and Huawei MindIE to shared registries with placeholder power/cost/TDP and new Huawei vendor color zones (OKLch/HSL and CIELab helpers), so Mock-ascend CI artifacts map through ETL instead of being dropped.The unofficial-run API now accepts
bmk_*per-config artifacts whenresults_bmkis missing (concatenates rows), and sequence handling gains8k/256and8k/625viaislOslToSequenceinstead of the old 1k/8k-only prefix hack.Inference charts get multi-sequence overlay (
MultiSequenceSelector,i_seq_extra,__seq<compact>synth hwKeys), a “Show as ingested” mode that merges unofficial points into the main scatter pipeline (mergeUnofficialIntoOfficial,i_uoff_ingested), and legend/color/reset fixes so new synth series stay visible. Branch-only defaults auto-pick 8K/256 and output throughput per GPU when an unofficial run loads without URL pins.Reviewed by Cursor Bugbot for commit c9c31a7. Bugbot is set up for automated code reviews on this repo. Configure here.